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i'HE  IMPACT  OF  THE  NEW  MEXICO  EXPERIMENTAL  MEDICAL  CARE  REVIEW  ORGANIZATION 
ON  THE  QUALITY  OF  THE  USE  OF  INJECTIONS 


Robert  H.  Brook  and  Kathleen  N.  Williams 


SUMMARY 


Evaluation  of  the  impact  of  formal  peer  review  of  the  New  Mexico 
Medicaid  program  during  its  first  two  years  of  operation  demonstrated 
that  peer  review  can  affect  the  level  of  quality  of  care  provided. 

Use  of  injections  dropped  by  over  60  percent . Major  decreases 
occurred  in  the  use  of  antibiotics  (e.g.,  linromvcin,  tetracycline, 
and  short-acting  penicillin);  certain  steroids;  and  antispasmodics. 

\ 

^Analysis  of  the  relationship  between  physician  character i st ics  and  the 
proper  use  of  injections  demonstrated  the  following:  (1)  being  a mem- 

ber of  a group  practice  that  billed  under  its  own  name  was  the  variable 
most  significantly  associated  with  the  proper  use  of  injections;  (2) 
for  physicians  billing  under  their  own  names,  being  board-certified 
was  the  principal  predictor  of  higher  quality  care;  (3)  provider  type 
(being  an  MD  rather  than  a DO)  and  specialty  were  also  significant 
predictors  of  quality;  (4)  foreign  medical  graduates  did  not  give  sub- 
stantially worse  care  than  did  U,S.  graduates;  and  (5)  the  6 percent 
of  physicians  who  gave  40  percent  of  the  medically  unnecessary  in- 
jections changed  their  behavior  dramatically  for  the  better. 


\ 


| tcctaitt  Hr  | 

lilt 

ID! 

njmroim 

mifiui 

until  Kctln  j/ 
fcm  s*c«M  □ 

eti  D 

101 

IT 

HiMSMW/tHIlMIlin  CMS 

1 

1'  Ml.  nt  or  SPECIAL 

t 

| 

PlSTRjBUTIon  r 

Approved  i( ; 
pisirll  uti< 


-V- 


prsgediic  pa^blank-mot  FI 


ACKNOWLEDGMENTS 


The  authors  wish  to  acknowledge  the  creative  leadership  of  the 
following  persons  in  New  Mexico  whose  efforts  led  to  the  peer  review 
activities  which  are  evaluated  in  this  paper:  Dr.  George  Boyden, 

Principal  Investigator  of  the  New  Mexico  Experimental  Medical  Care 
Review  Organization  grant;  Mr.  Richard  Heim,  director  of  New  Mexico's 
Department  of  Health  and  Social  Services;  Mr.  Robert  Flanagan  and 
Mr.  Wesley  Curtis  of  the  Dikewood  Corporation.  Ms.  Davy  Atelsek  and 
Dr.  Feather  Hair,  of  the  National  Center  for  Health  Services  Research, 
also  deserve  recognition  for  their  support  in  facilitating  this  eval- 
uation. Our  appreciation  is  also  extended  to  Dr.  John  Rolph  and 
Dr.  Charles  Phelps  for  their  assistance  with  the  statistical  analyses. 

The  research  reported  herein  was  supported  in  part  by  a Health 
Insurance  Grant  from  the  U.S.  Department  of  Health,  Education,  and 
Welfare  and  a contract  from  Health  Care  Management  Systems.  The  opin- 
ions and  conclusions  expressed  herein  are  solely  those  of  the  authors 
and  should  not  be  construed  as  representing  the  opinions  or  policies 
of  the  supporting  agencies. 

A similar  version  of  this  paper  appeared  in  The  Annals  of  Internal 
Medicine,  85:509-514,  October  1976. 


I.  INTRODUCTION 


Efforts  to  improve  the  quality  of  medical  care  are  here  to  stay. 

No  longer  are  they  confined  to  the  theorist;  instead,  they  are  man- 
dated by  two  federal  laws  (1,2)  and  one  national  professional  organi- 
zation, the  Joint  Commission  on  Accreditation  of  Hospitals  (3).  When 
fully  operational,  quality  assurance  activities  are  likely  to  consume 
2 to  5 percent  of  all  medical  care  expenditures,  or  $2  to  5 billion 
annually . 

Little  is  known  about  the  effectiveness  of  these  activities.  The 
amount  of  rhetoric  addressed  to  this  question  is  enormous  (4-8);  the 
amount  of  data  is  negligible.  For  instance,  most  studies  of  PSRO-like 
organizations  have  been  concerned  with  their  effect  on  cost  variables 
(e.g.,  cost  of  hospital  care),  not  on  quality  variables  (9-11).  One 
study  examined  the  effect  of  the  San  Joaquin  Foundation  for  Medical 
Care  on  the  utilization  of  ambulatory  services;  it  showed  small  but 
significant  improvements  (12).  In  general,  although  the  PSRO  program 
is  being  vigorously  implemented,  information  about  the  impact  that 
PSRO  or  PSRO-like  organizations  have  had  or  will  have  on  the  quality 
of  care  is  virtually  nonexistent. 

The  purpose  of  this  paper  is  twofold:  1)  to  describe  the  impact 

of  the  New  Mexico  Experimental  Medical  Care  Review  Organization  (a 
PSRO  prototype)  on  the  quality  of  the  use  of  injections  as  judged  by 
medical  criteria;  and  2)  to  relate  the  proper  or  improper  use  of  in- 
jections as  judged  by  medical  criteria  to  the  following  physician 
characteristics:  osteopathic  or  medical  doctor,  age,  practice  in  ur- 

ban or  rural  location,  specialty,  specialty  board  certification  status, 
and  foreign  or  U.S.  medical  graduate. 


i 
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II.  METHODS 


1 


The  Setting 

The  history  of  the  New  Mexico  Experimental  Medical  Care  Review 
Organization  (EMCRO)  has  been  extensively  documented  (13-16).  In  1971 
three  groups  (the  Health  and  Social  Services  Department  of  the  State 
of  New  Mexico,  the  New  Mexico  Foundation  for  Medical  Care,  and  the 
Dikewood  Corporation)  joined  together  to  develop  a system  for  pro- 
cessing claims  for  the  State's  Medicaid  program  and  for  reviewing  the 
quality  of  the  services  rendered  within  that  program. 

Claims  for  medical  services  rendered  are  submitted  by  the  phy- 
sician to  the  Dikewood  Corporation  (the  fiscal  intermediary  for  the 
Medicaid  program),  where  they  are  entered  into  a data  bank  and  screened 
on  the  basis  of  administrative  criteria  to  determine  whether  they  are 
valid.  Clerical  reviewers  (employees  of  the  New  Mexico  Foundation  for 
Medical  Care)  then  access  the  information  contained  on  these  claims 
and  compare  it  with  medical  guidelines  established  by  EMCRO  physicians. 
(A  copy  of  these  guidelines  is  available  elsewhere  [17].) 

If  a claim  passes  the  review,  it  is  returned  to  the  routine  pro- 
cessing system  for  payment.  If  a claim  fails  this  review,  it  is  held 
for  review  by  a practicing  physician  who  decides  whether  the  claim 
should  be  paid  or  denied  for  medical  reasons.  If  the  claim  is  denied, 
the  provider  may  appeal  the  decision  of  the  reviewing  physician. 


The  Population 

This  study  is  concerned  with  the  quality  of  care  given  to  the  New 
Mexico  Medicaid  population  between  September  1,  1971  (when  peer  review 
began)  and  August  31,  1973.  An  average  of  74,059  people  were  enrolled 
in  the  Medicaid  program  in  each  of  the  24  months  covered  by  this  study. 


Sixty  percent  of  the  population  remained  in  the  program  for  the  entire 
two  years.  Seventy-five  percent  of  the  population  was  covered  under 


the  Aid  to  Families  with  Dependent  Children  (AFDC)  category.  Of  the 


remainder,  the  majority  were  in  the  categories  of  Old  Age  Assistance 
or  Aid  to  the  Partially  and  Totally  Disabled;  relatively  few  were  in 
the  Aid  to  the  Needy  Blind  category. 
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Almost  one-half  of  the  population  was  14  years  of  age  or  younger; 
another  quarter  was  between  15  and  44  years  of  age;  the  remainder  was 
split  almost  evenly  between  ages  45  through  64  and  65  and  older.  Women 
comprised  almost  60  percent  of  the  entire  population.  The  population 
was  primarily  white  (81  percent);  the  next  largest  group  was  Indian 
(14  percent);  blacks  and  other  ethnic  groups  made  up  5 percent  of  the 
total.  One-third  of  the  population  lived  in  Bernalillo  County  (which 
includes  Albuquerque);  the  remainder  were  distributed  throughout  the 
state  with  no  other  county  having  more  than  6 percent  of  the  Medicaid 
populat ion . 


Study  Design 

In  order  to  perform  the  dual  objectives  of  this  study,  a data 
tape  was  obtained  from  the  New  Mexico  EMCRO  that  made  it  possible  to 
link  the  date  of  an  ambulatory  visit  with  provider  characteristics, 
specific  information  about  ambulatory  utilization,  and  facts  about 
quality  of  care  (such  as  whether  an  injection  was  given  and  whether  it 
was  denied  for  medical  reasons).  Physicians  in  New  Mexico  can  bill 
under  either  their  own  provider  number  or  a group  number.  Information 
about  age,  board  certification  status,  medical  school  of  graduation, 
specialty,  and  practice  location  was  obtained  from  each  provider  who 
billed  under  his  own  number  and  who  delivered  100  or  more  ambulatory 
visits  (home,  office,  emergency  department  and/or  outpatient  depart- 
ment) in  the  two-year  study  period. 

To  assess  the  impact  of  the  New  Mexico  EMCRO  on  the  use  of  in- 
jections, a controlled  time-series  design  was  used.  Controls  were 
the  number  of  prescriptions  and/or  laboratory  tests  given  per  Medicaid 
eligible  per  month,  since  virtually  no  review  for  the  medical  neces- 
sity of  these  services  was  made.  To  test  the  relationship  between  the 
proper  use  of  injections  and  physician  characteristics,  contingency 
table  analyses  were  performed.  (The  multivariate  analysis  supports 
the  conclusions  of  the  contingency  table  analysis  and  is  available 
from  the  authors  [17].) 


To  determine  the  relationship  between  physician  characteristics 
and  the  proper  use  of  injections,  a data  set  was  constructed  of  the 
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ambulatory  care  given  by  physicians  who  had  provided  LOO  or  more  am- 
bulatory visits  in  the  two-year  study  period.  The  360  providers  who 
met  this  criterion  provided  95  percent  of  the  ambulatory  visits  and 
94  percent  of  the  injections.  (Eleven  percent  of  all  injections  could 
not  be  attributed  to  any  one  provider.)  Of  these  360  providers,  47 
were  "group  providers"  and  billed  under  a group  name.  Of  the  313  phy- 
sicians who  billed  under  their  own  names,  81  were  Doctors  of  Osteo- 
pathy (DOs),  and  232  were  Doctors  of  Medicine  (MDs).  Additional  in- 
formation was  obtained  from  these  physicians:  age  from  295  (94  per- 

cent), county  of  practice  from  308  (98  percent),  board  certification 
status  from  292  (93  percent),  and  specialty  from  309  (99  percent). 

For  the  232  MDs,  data  on  school  (country)  from  which  they  graduated 
were  obtained  on  208  (90  percent).  Missing  data  were  usually  associ- 
ated with  a physician  who  had  died  or  moved  out  of  state  and  thus  had 
provided  relatively  few  services. 

Three  dependent  variables  were  constructed:  1)  the  number  of  in- 

jections billed  per  ambulatory  visit;  2)  the  number  of  injections 
denied  per  ambulatory  visit;  and  3)  the  number  of  injections  denied 
per  number  of  injections  billed.  Variable  1 reflects  the  propensity 
of  a physician  to  use  injections  (many  of  which  were  medically  unnec- 
essary). It  also  measures  the  effect  that  the  peer  review  process  had 
on  the  use  of  injections.  Variable  2 reflects  the  likelihood  of  a 
patient's  receiving  an  injection  judged  to  be  medically  unnecessary. 
Variable  3 measures  the  sanction  effect  of  the  peer  review  system. 


w 
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III.  RESULTS 


Injections  Billed  and  Denied  for  Medic al  Reasons 

General  Description.  Over  95,500  ambulatory  injections  were  given 
during  the  course  of  the  study;  of  these  nearly  50  percent  were  anti- 
biotics (Table  1).  During  the  study  period,  there  were  1,777,404  pa- 
tient eligible  months  and  334,608  ambulatory  visits.  Thus,  0.65  in- 
jections were  billed  per  eligible  per  year;  of  these,  0.19  injections 
were  denied  per  eligible  per  year.  On  the  average,  0.29  injections 
were  given  per  ambulatory  visit,  and  0.08  were  denied. 

Over  27,600  injections  (29  percent  of  those  billed)  were  denied 
for  medical  reasons  (Table  1).  Of  these,  98  percent  were  denied  for 
the  following  two  reasons:  type  of  injection  (93  percent)  did  not 

appear  warranted  from  the  information  on  the  claim,  and  the  frequency 
of  injection  (5  percent)  did  not  appear  warranted  from  information  on 
the  claim. 

Changes  Over  the  Study  Period.  The  New  Mexico  EMCRO  began  re- 
viewing the  use  of  injections  against  medical  guidelines  in  September 
1971;  during  the  period  September  1971  to  January  1972,  however,  little 
peer  review  activity  was  accomplished,  and  injection  usage  was  high. 
During  each  of  these  early  months,  about  9 percent  of  those  patients 
eligible  for  the  Medicaid  program  received  one  injection  (Figure  1); 
about  40  percent  of  all  ambulatory  visits  ended  with  an  injection 
(Figure  2).  In  January  1972,  members  of  the  EMCRO  staff  implemented 
educational  activities  consisting  of  discussions  with  small  groups  of 
physicians  who  were  using  injections  inappropriately  and  in  May  1972, 
formal  guidelines  concerning  the  proper  use  of  injectables  were  dis- 
tributed, published,  and  used  in  reviewing  claims. 

The  effect  of  these  two  actions  was  immediately  apparent.  The 
number  of  injections  billed  (Figures  1 and  2)  fell  substantially  after 
January  1972,  and  the  percent  of  the  number  of  injections  denied  per 
injection  billed  (Figure  3)  dramatically  increased  after  May  1972. 


TIME  (months) 


TIME  (months) 
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It  could  be  hypothesized  that  the  use  of  all  ambulatory  ser- 
vices by  the  Medicaid  population  was  falling  rapidly  during  this 
period,  and  that  the  EMCRO  did  not  cause  this  dramatic  change  in 
the  use  of  injections.  This  hypothesis  was  not  confirmed  by  the 
analysis  of  the  use  of  other  services  reviewed  only  occasionally 
or  not  at  all;  during  the  same  period,  increases  were  seen  in 
laboratory  services  (20  percent),  prescription  drugs  (1.2  per- 
cent), and  ambulatory  visits  (2.4  percent). 

Thus,  the  New  Mexico  EMCRO  had  two  outstanding  features 
with  respect  to  injections.  First,  it  prevented  almost  45,000 
unnecessary  injections,  thus  forestalling  iatrogenic  disease 
due,  for  instance,  to  inappropriate  use  of  antibiotics.  Second, 
by  the  end  of  the  study  period,  it  was  denying  payment  on  medi- 
cal grounds  for  40  percent  of  the  injections  still  being  given. 

Classes  of  Injections.  The  use  of  different  types  of  in- 
jections (e.g.,  lincomycin,  gold.  Vitamin  B^,,)  behaved  differently 
during  the  study  period.  Types  of  injections  were  grouped  into 
three  classes  depending  on  the  behavior  of  the  type  of  injection 
with  respect  to  the  number  billed  and  the  number  denied.  The 
first  class  of  injections  were  those  for  which  the  billing  per 
100  eligibles  decreased  remarkably  over  time  and  for  which  the 
proportion  of  injections  denied  rose  substantially.  Twelve 
different  types  of  injections  fell  into  Class  I (see  Table  1). 

This  class  includes  principally  the  antibiotics  other  than 
benzathine  penicillin  G (Bicillin),  the  steroids  other  than 
triamcinolone  acetonide  (Kenalog),  and  female  hormone  prepar- 
ations. The  pattern  for  the  use  of  this  type  of  injection  was 
typified  by  lincomycin:  the  number  of  injections  billed  per  100 

eligibles  (Figure  4)  dropped  from  1.7  in  January  1972  to  0.07  in 
August  1973;  the  percent  of  injections  denied  per  injections  billed 
rose  from  18  to  85  percent  (Figure  3).  Payment  was  denied  for  75  per- 


( months) 
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Table  1 


TOTAL  NUMBER  AND  PERCENT  OF  INJECTIONS  BILLED  AND  DENIED 
FOR  MEDICAL  REASONS,  BY  TYPE  AND  CLASS  OF  INJECTION 
NEW  MEXICO  MEDICAID  POPULATION 
1971-1973 


CLASS  AND 
TYPE  OF  INJECTION 

INJECTIONS  BILLED 

DENIED  FOR 
MEDICAL  REASONS 

NUMBER 

PERCENT 

(COLUMN) 

NUMBER 

PERCENT 

(ROW) 

CLASS  I 

1 

Penicillin 

I 

22,538 

23.6 

5,675 

25.2 

Lincomycin 

9,371 

9.8 

2,706 

28.9 

* 

Steroids  /ACTH 

5,704 

6.0 

2,696 

47.3 

Tetracycline 

3,412 

3.6 

1,388 

40.7 

Estrogen/Progee ter one 

3,115 

3.3 

1,934 

62.1 

Ampicillin 

1,800 

1.9 

463 

25.7 

Nonnarcotic  Analgesics 

1,306 

1.4 

465 

35.6 

Antibiotic/Nonantibiotic 

908 

0.9 

260 

28.6 

Synthetic  Ar.tispastnodics 

811 

0.8 

109 

13.4 

Gomenol 

672 

0.7 

456 

67.9 

Diuretics 

515 

0.5 

171 

33.2 

Antibiotic/ Antibiotic 

78 

0.1 

31 

39.7 

Other  than  Kanalog,  which  is  in  Class  II 
-4 

Percent  of  injections  billed 
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Table  1 - Continued 


DENIED  FOR 

INJECTIONS  BILLED 

1 MEDICAL 

i .. 

REASONS 

CLASS  AND 

PERCENT 

f 

( 

PERCENT 

TYPE  OF  INJECTION 

NUMBER 

(COLUMN) 

NUMBER 

(ROW) 

CLASS  II 

Tranquilizers 

3,785 

, 4.0 

1,303 

34.4 

Narcotic  Analgesics 

2,950 

3.1 

729 

24.7 

Vitamin  B^ 

2,743 

2.9 

! 2,070 

75.5 

* * 

Ant inause ants 

1,073 

1.1 

144 

13.4 

*** 

Ant ibis  t amines 

974 

1.0 

t 

384 

39.4 

Kenalog 

I 

847 

0.9 

301 

) 

35.5 

Iron 

385 

0.4 

158  ! 

41.0 

Other  injections 

15,253 

16.0 

4,381 

28.7 

CLASS  III 

Bicillin 

8,221 

8.6 

1,255 

15.3 

Tetanus 

i 

3,094 

i 

3.2 

30 

1.0 

DPT,  Flu,  Gaimna  Globulin 

2,743 

2.9 

101 

3.7 

Compnzine 

j 

1,780 

1 

1.9 

184 

10.3 

Phenergan 

804 

1 

0.8 

216 

26.9 

Gold 

693 

0.7 

28 

4.0 

Total  I 

95,575  1 

100.1 

27,638  ! 

28.9 

f 

Percent  of  injections  billed 


Other  than  Compazine,  which  is  in  Class  III 

•k'k'k 

Other  than  Phenergan,  which  is  in  Class  III 


NUMBER  OF  INJECTIONS  OF  LINCOMYCIN  BILLED  AND  DENIED  FOR 


9/71  10  11  12  1/72  2 3 4 5 6 7 8 9 10  11  12  1/73  2 

TIME  (months) 


-13- 


cent  or  more  of  the  following  injections  for  at  least  half  or  more  of 
the  study  period:  lincomycin,  tetracycline,  antibiotic-antibiotic 

combinations,  antibiotic-nonantibiotic  combinations,  ACTH,  estrogen, 
and  injectable  expectorant  (Gomenol).  Since  many  injections  in- 
cluded in  Class  I,  especially  the  antibiotics,  can  have  severe  side 
effects,  the  marked  decline  in  the  inappropriate  use  of  these  in- 
jections as  judged  by  medical  criteria  was  "clinically"  significant, 
insofar  as  potentially  harmful  side  effects  were  avoided. 

Class  II  injections  were  characterized  by  those  types  of  injec- 
tions with  a fairly  high  denial  rate  but  little  or  no  net  change  in 
the  billing  rate  during  the  two  years  of  the  study  period.  Eight 
types  of  injections  fell  into  this  category  (see  Table  1).  Although 
this  class  overlaps  with  Class  I for  steroids  and  analgesics,  it  is 
composed  chiefly  of  tranquilizers,  an  occasional  antihistamine  prep- 
aration, and  iron  and  Vitamin  , preparations.  Some  of  these  in- 
jections are  placebos;  although  payment  for  them  could  be  denied,  it 
apparently  was  difficult  to  prevent  them  from  being  used  as  such. 

For  instance,  over  75  percent  of  all  Vitamin  , injections  were 
denied  on  medical  grounds,  but  its  use  continued  to  rise  slightly. 

In  Class  III,  the  number  of  injections  billed  rose  over  time 
and  the  denial  rate  was  low.  Six  types  of  injections  were  included 
in  this  class  (see  Table  1).  The  immunizations,  most  of  which  were 
tetanus  given  for  acute  problems,  were  prominent.  The  injection 
policy  of  this  EMCRO  was  intended  to  encourage  the  use  of  long- 
acting  antibiotics  when  treating  conditions  such  as  streptococcal 
pharyngitis.  Apparently  this  was  accomplished,  as  judged  by  the 
increased  use  of  benthazine  penicillin  G for  this  condition.  Gold, 
which  is  used  in  the  treatment  of  rheumatoid  arthritis,  can  be  given 
only  by  injection;  the  pattern  for  gold  injections  billed  and  denied 
indicates  that  it  was  being  given  appropriately  for  that  diagnosis. 
Finally,  promethazine  hydrochloride  (Phenergan)  and  prochlorperazine 
(Compazine)  seem  to  be  the  therapies  of  choice  by  New  Mexico  phy- 
sicians for  their  respective  conditions.  These  two  drugs  had  a low 
denial  rate  and  the  number  of  these  injectables  billed  increased 
slightly  during  the  study  period.  On  the  basis  of  the  types  of 
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injections  included  in  Classes  I,  II,  and  III,  it  can  be  concluded 
that  peer  review  decisions  were  made  rationally  and  selectively. 


The  Relationship  ot  Injection  Utilization  and  Denial  to  Physician 
Characteristics 


Data  in  Tables  2-5  summarize  the  relationship  between  some  phy- 
sician characteristic  factors  and  utilization  and  denial  of  injections. 
The  three  major  dependent  variables  are  injections  billed  per  ambula- 
tory visit,  injections  denied  per  ambulatory  visit,  and  injections 
denied  per  injections  billed.  Because  of  the  large  numbers,  virtually 
all  categories  of  the  physician  characteristics  are  significantly  re- 
lated to  the  dependent  variables. 

Provider  Type.  Provider  type  had  a major  effect  on  the  proper 
use  of  injections,  in  that  the  dependent  variables  were  all  very  much 
lower  for  groups  than  for  either  MDs  or  DOs  (Table  2).  In  particular, 
the  injection  denial  rate  per  visit  for  groups  was  one-quarter  the 
rate  for  the  DO  population;  rne  denial  rate  per  injection  billed  for 
groups  was  two-thirds  the  rate  for  DOs.  The  rate  of  injections  billed 
per  ambulatory  visit  for  MDs  was  three-fifths  that  for  DOs;  their 
denial  rate  per  visit  was  not  quite  two-thirds  the  DO  rate. 

Age.  It  had  been  hypothesized  that  older  physicians  would  use 
more  injections  and  would  have  a higher  percentage  of  their  injec- 
tions denied  for  medical  reasons.  In  general,  this  hypothesis  was  not 
confirmed,  although  the  age  effect  was  complex  (Table  3).  Very  young 
MDs  (<34  years  of  age),  numbering  only  six  in  the  sample,  used  injec- 
tions more  inappropriately  than  did  any  other  MDs,  including  those  65 
and  older.  MDs  35  to  44  years  of  age  used  injections  most  appropri- 
ately. DOs  less  than  45  used  injections  more  appropriately  than  did 
those  45  and  older. 

County  of  Practice.  It  had  been  hypothesized  that  physicians 
in  all  rural  areas  of  the  state  would  have  a less  acceptable  level  of 
medically  appropriate  injections  than  physicians  in  the  more  urban 
areas.  This  hypothesis  was  not  confirmed.  Physicians  in  the  semi- 
urban  and  semi-rural  areas  appeared  to  practice  the  best  medicine. 

For  example,  the  number  of  injections  denied  per  ambulatory  visit  was 
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APPROPRIATENESS  OF  THE  USE  OF  INJECTIONS  BY  PROVIDER  TYPE  AND  PROVIDER  AGE 
NEW  MEXICO  MEDICAID  POPULATION.  1971-1973 


D.O. 

1 34 
35-44 
45-54 
55-64 

1 65 

Subtotal 


Total  | 295  212.123 

* 

Chi  square  significant  at  _ 0.05 
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0.05  and  0.06,  respectively,  for  the  semi -urban  and  semi-rural  areas, 
compared  with  0.09  and  0.12,  respectively,  for  the  urbanized  center 
and  the  rural  areas.  No  consistent  relationship  was  found  between 
location  of  practice  vis-h-vis  urban  and  rural  areas  and  the  three 
dependent  variables. 

Specialty . Host  providers  could  be  classified  into  one  of  the 
traditional  five  major  specialties  (internal  medicine,  pediatrics, 
general  practice,  obstetrics-gynecology,  and  general  surgery).  Data 
on  this  factor  (controlling  for  provider  type)  indicated  that  the  MD 
pediatricians  were  exemplary  physicians  and  that  the  DO  obstetrician- 
gynecologists  had  the  worst  record  in  the  use  of-  injections  (Table  4) . 
MD  internists  had  a slightly  lower  denial  rate  per  visit  than  did  gen- 
eral practitioners  (0.06  versus  0.07),  but  they  also  had  a higher  per- 
centage of  their  injections  denied  per  injection  billed  than  did  the 
general  practitioner  (0.36  versus  0.30). 

Specialty  Board  Certification.  The  relationship  between  board 
certification  status  and  the  appropriateness  of  the  use  of  injections 
was  also  examined.  Data  referred  only  to  those  physicians  who  had  ob- 
tained certification  by  one  American  specialty  board  vis-h-  'is  those 
physicians  who  had  not.  It  had  been  hypothesized  that  board  certifi- 
cation status  would  be  strongly  related  to  the  appropriate  use  of  in- 
jectable drugs,  and  in  fact  this  was  confirmed  (Table  5).  Noncerti- 
fied  physicians  (both  DOs  and  MDs)  used  more  injections,  had  more  in- 
jections denied  for  medical  reasons,  and  had  a higher  percentage  of 
their  injections  denied.  In  the  case  of  the  number  of  injections 
denied  per  ambulatory  visit,  the  value  was  threefold  higher  for  the 
noncertified  DO  than  for  his  certified  colleague;  the  respective  dif- 
ference for  MDs  was  fourfold. 

Figure  5 highlights  the  differences  in  injections  billed  by  phy- 
sician type  and  board  certification  status.  The  number  of  injections 
billed  per  ambulatory  visit  declined  markedly  for  all  four  categories 
of  physicians,  beginning  in  January  1972.  Similarly,  the  number  of 
injections  billed  that  were  subsequently  denied  increased  for  all 
classes  of  physicians  after  May  1972.  At  the  start  of  the  study,  the 
number  of  injections  denied  per  injection  billed  was  between  0.15  and 
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APPROPRIATENESS  OF  THE  USE  OF  INJECTIONS  BY  PROVIDER  TYPE  AND  SELECTED  SPECIALTIES 
NEW  MEXICO  MEDICAID  POPULATION , 1971-1973 


Number  of 

1 

* ! 

Injections 

. ■ 

Inject  ions 

h 

Inj  ect Ions 

Billed  per 

Denied  per 

Denied  per 

Provider  Type 

Ambulatory 

Injections 

Inject  ions 

Ambul  atory 

Ambulatory 

Injections 

and  Specialty 

Providers 

Visits 

Billed 

Denied 

Visit 

Visit 

Billed 

M.D. 

Internal 

l 

! 

medicine  1 

31 

16,106 

2,683 

963 

0.  17 

0.06 

0.36 

Pediatrics  j 

23 

11,104 

1,501 

154 

0.14 

0.01 

0.10 

General 

Practice 

81 

61,566 

14,403 

4,356 

0.23 

0.07 

0.30 

OB-Gyn 

16 

3,580 

444 

194 

0.12 

0.05 

0.44 

Surgery 

61 

18.227 

3,627 

1,610 

0.2C 

0.09 

0.44 

Subtotal 

212 

110,583 

22,658 

7,277 

0.20 

0.07 

0.32 

D.O. 

Internal 

medicine 

2 

1,524 

111 

17 

0.07 

0.01 

0.15 

Pediatrics 

2 

2,586 

570 

133 

0.22 

0.05 

0.23 

General 

practice 

45 

70,001 

23,174 

6,330 

0.33 

0.09 

0.27 

OB— Gyn 

10 

9,985 

5,721 

2,295 

0.57 

0.23 

0.40 

Surgery 

15 

12,919 

4,436 

1,272 

0.34 

0.10 

0.29 

Subtotal 

74 

97,015 

34,012 

10,047 

0.35 

0.10 

0.30 

Total 

286 

20/, 598 

56,670 

17,324 

0.08 

0.31 

* 

Chi  square  significant  at  ^ 0.05 
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0.25  for  the  four  classes  of  physicians.  After  May  1972,  the  figure 
jumped  to  about  0.55  for  noncertified  DOs  and  MDs  and  to  not  quite 
0.35  for  certified  physicians. 

Certain  trends  are  apparent  from  these  data  and  from  Figure  5. 

In  the  "before"  period  (9/71  to  1/72),  the  DOs  (both  certified  and 
noncertified)  used  injections  more  inappropriately  than  did  either  MD 
group.  In  the  "after"  time  period,  however,  the  noncertified  DOs  be- 
gan to  behave  more  like  their  noncertified  MD  counterparts  than  like 
their  certified  DO  colleagues.  Similarly,  the  certified  DOs  began  to 
practice  more  like  their  certified  MD  colleagues.  Thus,  in  the  "before" 
period,  the  more  important  explanatory  variable  (of  the  two  under  dis- 
cussion) was  provider  type;  by  the  end  of  the  study  period,  it  was 
board  certification  status. 

Foreign  and  U. S. -Educated  Physicians.  The  last  analysis  concerned 
only  MDs  and  related  country  of  medical  graduation  to  the  dependent 
variables.  It  had  been  hypothesized  that  foreign  medical  graduates 
(FMGs)  would  have  a higher  incidence  of  inappropriate  use  of  injec- 
tions than  would  U.S.  medical  graduates  (USMGs) . The  data  confirmed 
this  difference,  but  it  was  not  a particularly  striking  one.  For  ex- 
ample, USMGs  had  a billed  per  visit  rate  of  0.19,  and  (non-Canadian) 

FMGs  had  a rate  only  slightly  higher,  0.22.  Similarly,  the  denied  per 
visit  rate  for  USMGs  was  0.06  and  0.08  for  FMGs;  the  rate  of  injec- 
tions denied  per  injection  billed  was  0.31  and  0.36  for  USMGs  and  FMGs, 
respectively.  Thus,  the  hypothesized  effect  ot  a much  higher  rate  of 
denial  for  FMGs  than  for  USMGs  was  not  found  in  this  study. 

Summary  of  the  Contingency  Table  Analysis.  Two  major  hypotheses 
were  substantiated  by  this  analysis:  DOs  had  a higher  rate  of  inap- 

propriate use  of  injections  than  did  MDs,  and  non-board-certified  phy- 
sicians had  a higher  rate  of  inappropriate  use  than  certified  physi- 
cians. By  the  end  of  the  study,  the  latter  effect  predominated.  Two 
hypotheses  were  not  supported.  Although  USMGs  had  a slightly  lower 
rate  of  inappropriate  use  of  injections  than  did  their  FMG  counter- 
parts, the  difference  was  not  impressive  (and  was  not  statistically 
significant  in  the  multivariate  regression  analyses).  Similarly,  the 
inappropriate  use  of  injections  increased  with  age  in  the  DO  popula- 
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tion,  but  the  MD  data  suggested  that  the  very  young  physicians  used 
injections  most  inappropriately.  Unexpected  findings  were  that  the 
MD  pediatrician  used  injections  most  appropriately,  that  the  DO  obste- 
trician-gynecologists used  injections  most  inappropriately,  and  that 
MD  internists  and  MD  general  practitioners  did  not  differ  substantially 
in  their  use  of  injections. 

The  Outlier  Physician.  One  additional  question  was  explored, 
namely,  whether  a given  class  of  physicians  as  a whole  had  a higher 
rate  of  inappropriate  injections  or  whether,  in  contrast,  only  certain 
physicians  within  that  class  were  responsible  for  the  high  rate. 

The  360  providers  were  divided  into  the  three  provider  classifi- 
cations (MDs , DOs , and  groups)  and  rank  ordered  by  two  variables: 
the  number  of  ambulatory  visits  and  the  number  of  injections  denied. 
Cumulative  percentages  of  ambulatory  visits,  injections  billed,  and 
injections  denied  were  calculated.  Taking  the  rank  order  of  ambula- 
tory visits  first,  those  MDs  and  DOs  who  gave  a large  number  of  ambu- 
latory visits  did  not  have  a disproportionate  number  of  injections,  as 
measured  by  the  rate  of  injections  either  billed  or  denied.  The  top 
14  MDs,  who  gave  10.2  percent  of  the  ambulatory  visits,  gave  8.5  per- 
cent of  the  injections  and  had  only  8.9  percent  of  the  injection  de- 
nials. For  the  top  5 DOs,  the  figures  were  10.7  percent,  11.6  percent, 
and  8.0  percent,  respectively.  Thus,  one  could  conclude  that  those 
providers  who  gave  a large  part  of  the  ambulatory  services  to  this 
population  were  not  responsible  for  the  bulk  of  unnecessary  injections. 

When  providers  were  rank  ordered  on  injections  denied,  striking 
relationships  emerged.  The  top  five  MDs,  who  produced  11.5  percent  of 
the  injections  denied,  had  only  2 percent  of  the  ambulatory  visits. 
Likewise,  the  top  three  DOs,  who  gave  11.1  percent  of  the  injections 
denied,  had  only  2.4  percent  of  the  ambulatory  visits.  The  22  pro- 
viders (6  percent)  in  the  top  two  classes  of  MDs  and  DOs  were  re- 
sponsible for  40.9  percent  of  the  inappropriate  injections,  29.0  per- 
cent of  the  injections  billed,  and  only  13.5  percent  of  the  ambulatory 
visits.  Of  the  360  providers  studied,  these  22  providers  (6  percent) 
gave  over  40  percent  of  the  inappropriate  injections  in  the  Medicaid 
program.  However,  the  outlier  physicians  changed  their  behavior  sub- 
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stantially  as  a function  of  the  peer  review  system;  their  injection 
patterns  more  closely  resemble  the  injection  patterns  of  their  col- 
leagues at  the  end  of  the  study  than  at  the  beginning.  In  January, 
1972,  the  outlier  MDs  gave  0.82  injections  per  visit,  the  outlier  DOs 
gave  0.91,  and  all  remaining  physicians  gave  0.32  injections  per  visit; 
respective  values  for  August  1973  were  0.31,  0.49,  and  0.19. 
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IV.  DISCUSSION 


This  paper  describes  an  evaluation  of  an  operational  quality  as- 
surance program  that  was  shown  to  work.  The  New  Mexico  EMCRO  was  able 
to  correct  a substantial  problem  in  the  use  of  injections,  through 
education  of  physicians  who  were  using  injections  inappropriately  and 
through  denial  of  payment  for  services  rendered.  It  seems  clear  that 
many  iatrogenic  complications  (both  local  and  systemic)  in  the  use  of 
injections  in  general  and  antibiotics  in  particular  were  prevented, 
through  the  elimination  of  over  60  percent  of  the  injections  in  a two- 
year  period.  Moreover,  virtually  all  physicians  responded  to  this 
peer  review  system.  Most  impressive  were  the  dramatic  changes  in  the 
behavior  of  the  6 percent  of  physicians  who  were  responsible  for  40 
percent  of  the  inappropriate  injections.  No  complex  statistical  anal- 
ysis is  needed  to  confirm  the  positive,  clinically  relevant  effect  that 
the  EMCRO  had  on  the  quality  of  care  given  to  Medicaid  patients. 

This  assertion  is  tempered  by  the  knowledge  that  improvement  in 
the  use  of  injections  was  the  major  (and  perhaps  the  only  clinically 
important)  effect  produced  by  the  EMCRO  in  the  first  three  years  of 
its  operation.  If  the  PSRO  model  is  eventually  to  prove  successful  in 
substantially  improving  the  health  of  the  American  people,  then  the 
quality  of  care  for  other  services  must  be  measured  and  improved. 
Whether  the  New  Mexico  success  can  be  replicated  for  other  services  in 
this  and  other  states  is  not  known.  Nevertheless,  efforts  are  war- 
ranted to  determine  if  the  improper  use  of  injections  exists  in  other 
population  groups  (not  necessarily  limited  to  Medicaid  groups)  in  other 
states.  If  so,  corrective  steps  similar  to  those  employed  by  the  New 
Mexico  EMCRO  should  be  implemented,  at  least  on  an  experimental  basis. 

Those  physicians  who  gave  the  most  ambulatory  services  to  this 
Medicaid  population  were  not  the  ones  who  used  injections  most  inappro- 
priately; as  judged  by  this  criterion,  they  were  not  providing  inferior 
care.  Publication  of  the  names  of  physicians  who  deliver  the  most 
Medicaid  services  often  carries  with  it  the  implication  that  they  are 
delivering  inferior  (or  perhaps  even  fraudulent)  care;  the  data  re- 
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ported  above,  however,  do  not  support  this  notion.  In  the  absence  of 
any  other  persuasive  reason  for  such  publication,  therefore,  the  prac- 
tice might  usefully  be  stopped  or  curtailed  unless  future  studies  in- 

1 8 

dicated  that  these  physicians  were  indeed  giving  lower  quality  of  care. 

The  results  of  this  study  are  limited  in  that  they  relate  only  to 
a few  hundred  physicians  delivering  care  to  the  Medicaid  population  of 
one  rural  Western  state.  Nevertheless,  a few  important  findings  stand 
out.  First,  medical  practice  in  a partnership,  group,  or  clinic  ar- 
rangement was  the  most  significant  variable  in  predicting  the  proper 
use  of  injections.  Second,  for  individual  providers,  the  most  impor- 
tant variables  in  predicting  appropriate  use  of  injections  were  spe- 
cialty board  certification,  provider  type  (being  an  MD),  and  specialty 
(being  an  MD  pediatrician).  Third,  as  judged  by  this  one  measure 
(proper  use  of  injections),  DO  obstetrician-gynecologists  and  young 
MDs  (less  than  35  years  of  age)  used  injections  quite  inappropriately. 
Fourth,  the  care  given  by  internists  and  general  practitioners  did  not 
differ  substantially.  Fifth,  foreign  medical  graduates  did  not  give 
substantially  worse  care  than  did  U.S.  medical  graduates.  Sixth,  site 
of  practice  (rural  or  urban)  did  not  predict  higher  or  lower  quality 
of  care. 

Were  these  findings  to  be  substantiated  in  different  populations 
on  larger,  more  representative  samples,  and  using  a greater  variety  of 
indicators  of  quality  of  care,  reorientation  of  certain  policies  with 
respect  to  quality  of  health  care  might  be  in  order.  For  example, 
attempts  to  require  relicensure  and  recertification  of  older  physi- 
cians should  include  efforts  to  upgrade  qualifications  for  initial 
entrance  to  the  practice  of  medicine.  Major  deficiencies  exist  in  the 
quality  of  care  provided  and  these  seem  not  to  be  related  to  the  ac- 
quisition of  complex  new  knowledge.  Thus,  relicensure  and  recert if i- 
cation  might  be  related,  at  least  in  the  beginning,  to  the  performance 
of  relatively  simple  and  basic  medical  practices  and  be  directed  at 
those  providers  who  do  not  perform  these  activities  well.  Although 
much  more  study  of  the  field  is  needed,  relicensure  that  is  focused 
principally  on  acquisition  of  new  knowledge  in,  say,  sophisticated 
therapeutic  techniques,  without  addressing  problems  in  basic  medical 
care  (such  as  injections),  would  seem  to  be  misdirected. 
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The  role  of  patient  expectations  in  influencing  physician  deci- 
sions needs  to  be  clarified.  Insofar  as  patients  do  not  expect  in- 
jections for  pediatric  illnesses  (immunizations  excepted)  but  do  ex- 
pect injections  for  problems  of  old  age  or  "female  conditions,"  a 
bias  for  better  or  poorer  quality  of  care  being  delivered  by  spe- 
cialists concerned  with  one  or  the  other  of  these  population  groups 
exists.  There  is  a degree  of  patient  responsibility — and  need  for 
patient  education — that  cannot  be  overlooked  in  this  context. 

Those  features  of  the  large  group  practices  that  encouraged  the 
proper  use  of  injections  should  be  identified  and,  if  possible, 
transferred  to  the  solo  fee-for-service  setting.  Since  group  prac- 
tices had  a better  record  than  individual  practitioners  at  the  out- 
set of  the  study  as  well  as  at  the  end,  the  EMCRO  had  relatively  less 
impact  in  that  setting;  by  implication,  the  group  practice  had  cer- 
tain intrinsic  characteristics  that  promoted  the  more  appropriate  use 
of  injectables.  Whether  this  is  an  outcome  related  to  characteristics 
of  physicians  within  the  group  (e.g.,  specialty  or  board  certification 
status)  or  to  organizational  and  operational  factors  (or  to  a seren- 
dipitous combination  of  all  these  features)  is  a question  worthy  of 
substantial  empirical  investigation. 

Finally,  that  this  evaluation  occurred  at  all  must  be  emphasized. 
Its  accomplishment  required  the  cooperation  of  a Foundation  for  Medi- 
cal Care,  physicians,  a fiscal  intermediary,  a state  department  of 
health,  various  government  agencies,  and  a research  organization — a 
combination  rarely  encountered  in  the  past.  Efforts  at  evaluating 
quality  of  care  can  generate  a host  of  fears;  leadership  from  many 
and  diverse  groups  will  be  needed  if  evaluation  of  activities  such  as 
PSROs  is  to  be  successful.  Without  objective  evaluation  with  appro- 
priate feedback  to  the  operational  quality  assurance  agency,  a quality 
assurance  program  like  PSRO  may  fail  to  achieve  its  objective  of  im- 
proving the  health  of  the  American  people.  The  successful  collabora- 
tion represented  by  this  study  is  a positive  step  toward  that  objec- 
t ive . 
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